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Flight  Inspection  Crew  Resource  Management 
Training  Needs  Analysis 


The  training  of  aircrews  has  changed  markedly 
from  the  early  days  of  aviation.  Initially,  the  emphasis 
of  air  crew  training  focused  on  developing  the  technh 
cal  skills  of  the  individual  crew  members.  The  under¬ 
lying  assumption  was  that  if  crew  membej^s  were 
technically  proficient  at  their  respective  jobs,  they 
would  automatically  be  able  to  operate  effectively  as  a 
crew  (for  historical  reviews,  see  Driskell  &  Adams, 
1992;  FAA,  1993,  Hartel  &  Hartel,  1995;  Helmreich 
&  Foushee,  1993).  During  the  1970s,  however,  evi¬ 
dence  from  airline  accident  reports,  flight  simulator 
transcripts,  and  interviews  with  crew  members  sug¬ 
gested  that  the  above  assumption  was  inherently  flawed. 
Technical  competence,  by  itself,  did  not  ensure  a 
successful  mission.  Instead,  mission  success  was  de¬ 
pendent  on  the  manner  in  which  technically  compe¬ 
tent  crews  coordinated  their  individual  efforts  (Cooper, 
White  &  Lauber,  1979;  Lauber  1987;  and  Ruffel 
Smith,  1979).  Training  to  develop  aircrew  coordina¬ 
tion  skills  became  known  as  Crew  Resource  Manage¬ 
ment  or  simply  CRM  (FAA,  1993).  More  recently, 
organizational  factors  (in  particular  those  which  de¬ 
termine  the  consequence  of  performance,  and  those 
which  provide  flight  crews  with  the  resources  neces¬ 
sary  to  perform  their  jobs)  have  also  been  found  to  be 
important  determinants  of  aircrew  performance  (Hack- 
man,  1993).  These  findings  suggest  that  aircrew  per¬ 
formance  depends  on  at  least  three  factors:  (1)  technical 
competencies,  (2)  crew  resource  management  skills, 
and  (3)  the  organizational  context  in  which  crews 
perform. 

Issues  surrounding  the  above  performance  factors 
were  identified  in  a  National  Transportation  Safety 
Board  (NTSB)  report  of  the  October  26,  1993,  fatal 
crash  of  a  Federal  Aviation  Administration  (FAA) 
flight  inspection  aircraft,  N82  (NTSB,  1994).  This 
was  the  second  fatal  flight  inspection  aircraft  accident 
in  five  years.  In  reviewing  the  factors  contributing  to 
the  October  crash,  the  NTSB  issued  one  urgent  action 
and  seven  priority  action  recommendations  to  the 
FAA.  Included  in  the  latter  was  the  recommendation 


to  institute  Crew  Resource  Management  (CRM)  train¬ 
ing,  as  outlined  in  the  FAA  CRM  Advisory  1 20-5 1 ,  at 
each  of  Flight  Inspection  Area  Office  (FIAO). 

Prior  to  the  accident  of  N82,  the  FAA  was  already 
in  the  final  stages  of  developing  a  CRM  training 
course  for  all  FAA  aircrews.  With  the  advent  of  the 
accident,  the  FAA  Administrator  elevated  the  priority 
of  this  initiative,  and  a  CRM  task  force  led  by  the  Civil 
Aeromedical  Institute  (CAMI)  was  created  to  guide 
the  process  of  developing  a  CRM  course  specifically 
for  the  flight  inspection  mission.  Also  included  in  the 
CRM  task  force  was  the  FAA’s  Senior  Flight  Safety 
Officer  and  representation  from  six  FIAOs. 

One  of  the  first  steps  taken  by  the  CRM  task  force 
was  to  conduct  a  CRM  training  needs  analysis  based 
on  issues  addressed  during  the  November  1993  safety 
meetings  conducted  at  each  FIAO.  These  meetings 
were  mandated  by  the  FAA  Administrator  in  response 
to  the  accident  of  N82.  The  purpose  of  the  meetings 
was  to  identify  organizational  and  crew  issues  that 
impacted  the  safety  of  all  FAA  flight  operations. 

“Flight  inspection”  refers  to  the  airborne  tests  con¬ 
ducted  to  ensure  that  navigational  aids  are  sending 
accurate  signal-in-space  guidance,  and  to  ensure  that 
instrument  flight  procedures  are  accurate  and  will 
safely  guide  aircraft  to  their  destination.  A  flight 
inspection  crew  consists  of  a  pilot  in  command  (PIC), 
a  second  in  command  (SIC;  co-pilot),  and  an  elec¬ 
tronics  technician  (ET).  The  flight  inspection  mis¬ 
sion  differs  from  other  forms  of  flying  (such  as  air 
transport)  in  that  most  flight  maneuvers  are  con¬ 
ducted  within  the  terminal  area,  at  low  altitudes,  and 
at  times  running  counter  to  the  established  air  traffic 
flow  pattern.  This  requires  a  high  degree  of  traffic 
vigilance,  as  well  as,  coordination  between  the  flight 
inspection  crew,  air  traffic  control,  and  a  ground 
electronics  technician. 

Although  generic  CRM  programs  modeled  after 
commercial  airlines  have  been  developed,  the  FAA’s 
CRM  Advisory  Circular  (120-51  A)  warns  against 
using  such  programs  without  first  customizing  them 
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to  reflect  the  nature  and  needs  of  the  specific  flying 
mission.  Following  that  guidance,  a  CRM  training 
needs  analysis  was  conducted  to  ensure  that  the  mis¬ 
sion-specific  concerns  of  the  flight  inspection  crews 
were  addressed  in  their  CRM  awareness  training. 
Specifically,  the  focus  of  the  needs  analysis  was  to 
identify:  (1)  the  phases  of  the  flight  inspection  mis¬ 
sion  that  were  most  problematic  from  a  CRM  per¬ 
spective,  (2)  the  CRM  dimensions  that  needed  to  be 
especially  emphasized  in  a  CRM  awareness  course, 
and  (3)  organizational  factors  that  needed  to  be  ad¬ 
dressed  to  support  a  long  term  CRM  initiative. 

This  report  presents  the  results  of  the  needs  analy¬ 
sis,  along  with  a  discussion  of  the  CRM  training 
implications  that  emerged.  However,  before  proceed¬ 
ing,  the  reader  may  find  helpful  the  following  back¬ 
ground  information  about  CRM,  the  FAA’s  prior 
CRM  training  initiatives,  and  information  concern¬ 
ing  the  flight  inspection  aircraft  accidents  that  high¬ 
lighted  the  need  for  flight  inspection  CRM  awareness 
training. 

CRM  Background 

Crew  Resource  Management  (CRM)  refers  to  the 
“effective  utilization  of  all  available  resources-hard- 
ware,  software,  and  personnel-to  achieve  safe,  effi¬ 
cient  flight  operations”  (p.  8,  Driskill  &  Adams, 
1992).  CRM  differs  from  other  forms  of  team  work 
training  both  in  the  nature  of  decisions  that  must  be 
made,  as  well  as  the  consequences  of  those  decisions. 
Typically,  aircrews  must  make  CRM  related  deci¬ 
sions  in  a  matter  of  seconds  and  the  results  of  those 
decisions  have  life  and  death  consequences.  Although 
CRM  training  began  with  aviation,  it  now  has  ex¬ 
tended  into  such  other  fields  as  operating  rooms 
(Howard,  Gaba,  Fish,  Yang,  &  Sarnquist,  1992), 
nuclear  power  plants  (Gaddy  &  Wachtel,  1992),  and 
combat  units  (Andrews,  Wagg  &  Bell,  1992). 

The  need  for  specialized  training  in  CRM  stems 
from  the  developmental  tract  that  many  technical 
fields  require  of  their  professionals.  For  the  most  part, 
the  emphasis  of  technical  training  is  on  the  develop¬ 
ment  of  individual  knowledge,  skills,  and  abilities. 
Teamwork  is  often  represented  as  the  simple  aggrega¬ 
tion  of  individual  efforts.  Because  of  the  lack  of 
formal  training  in  teamwork,  technically  proficient 


individuals  may  have  well  developed  strategies  for 
working  by  themselves,  but  these  same  strategies  may 
not  be  the  most  effective  for  use  in  a  group  (Salas, 
Dickinson,  Converse,  &c  Tannenbaum,  1992).  For 
example,  persons  who  are  used  to  making  decisions  on 
their  own  may  not  take  the  time  to  inform  others  of 
the  rationale  behind  those  decisions.  Whereas  the 
individual  may  understand  what  is  going  on,  others 
may  have  only  a  limited  view  of  the  situation,  a  view 
that  can  also  be  colored  by  their  own  perspective. 

One  of  the  main  emphases,  then,  of  CRM  training 
is  to  develop  the  resource  management  skills  necessary 
to  ensure  that  all  group  members  are  operating  from 
a  common  frame  of  reference,  and  that  this  reference 
is  consistent  with  what  is  actually  occurring.  Specific 
skills  developed  in  CRM  awareness  training  com¬ 
monly  include:  (1)  communication  skills,  such  as 
inquiry,  advocacy,  and  assertion;  (2)  methodologies 
for  identifying  problems  and  making  decisions  under 
severe  time  constraints;  (3)  self-monitoring  skills  for 
critiquing  decisions  and  actions  of  the  crew;  (4)  con¬ 
flict  resolution  skills;  (5)  skills  associated  with  crew 
leadership,  followership,  and  concern  for  the  task;  (6) 
interpersonal  skills  necessary  for  maintaining  a  pro¬ 
fessional  crew  climate;  (7)  situational  awareness  and 
distraction  avoidance  skills;  (8)  workload  planning 
and  distribution  skills;  and  (8)  identifying  personal 
stressors  and  developing  effective  stress  reduction 
techniques  (FAA,  1993). 

As  an  organizational  intervention,  CRM  typically 
involves  a  three-phase  approach  (FAA,  1993).  This 
includes  formal  classroom  training,  practice  and  feed¬ 
back,  and  organizational  reinforcement.  In  the  first 
phase,  theawareness phase,  CRM  principles  are  taught 
in  a  classroom  setting  and  are  reinforced  through  the 
use  of  accident  recreations,  case  history  discussions, 
role-playing  exercises,  etc.  The  purpose  of  this  train¬ 
ing  is  to  provide  crew  members  with  a  common 
understanding  of  how  CRM  principles  apply  to  their 
mission,  and  to  practice  some  basic  skills.  Once  gen¬ 
eral  principles  are  acquired,  they  are  further  refined 
during  the  second  phase  of  training,  practice  and 
feedback.  Ideally,  this  phase  of  training  should  be 
conducted  on  an  annual  basis  in  a  simulated  mission 
training  environment.  Crews  then  perform  mission 
oriented  tasks  while  being  video-taped.  Afterwards, 
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crew  members  view  the  video  playback  of  their  perfor¬ 
mance  and  perform  a  self  critique  of  their  CRM  skills. 
Finally,  in  the  third  phase,  ^^reinforcement phase ^  the 
organization  ensures  that  CRM  principles  become 
part  of  the  organization’s  culture  by  adopting  policies, 
procedures,  and  practices  that  are  consistent  with,  and 
reinforce,  CRM  principles.  Among  other  actions,  this 
involves  incorporating  CRM  principles  in  all  forms  of 
training,  requiring  managers  to  become  effective  role 
models  for  CRM  behaviors,  and  establishing  CRM 
criteria  for  use  in  selection,  promotion,  and  perfor¬ 
mance  appraisal  decisions.  For  a  more  complete  re¬ 
view  of  CRM  training  and  its  history,  see  Driskell  & 
Adams  (1992),  Hartel  &  Hartel  (1995),  and  Wiener, 
Kanki  &  Helmreich  (1993). 

FAA  CRM  Training 

Two  flight  inspection  CRM  training  initiatives 
preceded  the  accident  of  N82.  In  early  1990,  CRM 
awareness  training  was  delivered  to  all  flight  inspec¬ 
tion  pilots.  The  course  was  developed  and  conducted 
by  United  Airlines.  Although  the  course  exposed 
participants  to  CRM  principles,  it  was  not  custom¬ 
ized  to  the  mission  needs  of  flight  inspection  crews. 
Furthermore,  electronic  technicians  were  not  included 
in  the  training.  Finally,  the  course  itself  was  delivered 
as  a  stand-alone  program,  with  no  formal  attempts 
made  to  integrate  it  into  a  long-term  CRM  program 
as  described  in  the  previous  section. 

To  address  the  above  weakness,  in  1992,  under  the 
sponsorship  of  the  FAA’s  Flight  Safety  Program,  a 
CRM  initiative  began,  with  the  goal  of  developing  a 
long-term  CRM  training  effort  for  each  of  the  five 
FAA  aircraft  programs:  (1)  flight  inspection,  (2)  flight 
standards,  (3)  Washington  flight  program,  (4)  re¬ 
search  and  development,  and  (5)  training  academy. 
The  Flight  Safety  Program  targeted  the  flight  inspec¬ 
tion  aircrews  as  having  the  most  pressing  need  for 
CRM  training.  Thus,  work  began  to  develop  a  proto¬ 
type  flight  inspection  CRM  training  effort  that  would 
serve  as  a  guide  for  the  remaining  aircraft  programs. 
Guiding  this  effort  was  an  informal  taskforce,  consist¬ 
ing  of  the  FAA  Senior  Flight  Safety  Officer,  CAMI, 
and  representation  from  flight  inspection  crews. 
Throughout  1993,  the  taskforce  reviewed  existing 
aircarrier  CRM  programs,  as  well  as  the  results  from 


published  CRM  research.  With  the  advent  of  the  N82 
accident,  a  formal  CRM  taskforce  was  developed  and 
charged  to  deliver  flight  inspection  CRM  awareness 
training  during  the  second  quarter  of  fiscal  year  1 994. 
The  new  taskforce  agreed  to  developed  a  one-day 
post-accident  CRM  awareness  training  course  which 
would  be  followed  by  a  three-day  course  following  the 
completion  of  a  CRM  needs  analysis. 

Flight  Inspection  Aircraft  Accidents 

Although  CRM  training  can  be  viewed  as  a  proac¬ 
tive  means  for  an  organization  to  ensure  the  safety  of 
its  flight  operation,  some  organizations  wait  until 
they  have  experienced  an  aircraft  accident  before  they 
initiate  a  CRM  training  program  (Helmreich  &: 
Foushee,  1993).  Described  below  are  the  three  flight 
inspection  aircraft  accidents  (two  fatal  and  one  non- 
fatal)  that  preceded  the  development  of  the  FAA’s  CRM 
awareness  training  for  its  flight  inspection  aircrews. 
Information  concerning  these  accidents  is  documented 
in  the  NTSB  accident  report  on  N82  (NTSB,  1994). 

On  October  26,  1993,  N82,  a  Beech  Super  King 
Air  300/F,  crashed  into  mountainous  terrain  near 
Front  Royal,  Virginia.  All  three  crew  members  died  in 
the  crash.  The  plane  was  owned  by  the  FAA  and 
operated  by  the  Atlantic  City,  New  Jersey,  Flight 
Inspection  Area  Office  (FIAO).  The  NTSB  deter¬ 
mined  that  the  most  probable  cause  of  the  accident 
was  the  failure  of  the  pilot-in-command  to  operate 
under  visual  flight  rules  over  mountainous  terrain. 
Subsequent  investigations  conducted  by  the  NTSB 
revealed  that  the  PIC  had  a  history  of  ignoring  visual 
flight  rules,  failing  to  adhere  to  checklists,  and  not 
acknowledging  inquiries  by  the  second-in-command 
when  such  matters  were  brought  to  his  attention. 
Furthermore,  when  crew  members  reported  these  in¬ 
fractions  to  the  Atlantic  City  flight  operations/sched- 
uling  supervisor  (FO/SS),  the  supervisor  failed  to  take 
corrective  action  (NTSB,  1994). 

In  addition  to  the  formal  report  on  N82,  two  earlier 
flight  inspection  aircraft  accidents  also  involved  prob¬ 
lems  with  situational  awareness,  communications, 
and  overall  crew  coordination.  On  November  2, 1988, 
a  Rockwell  Jet  Commander  (1 121  A)  operated  by  the 
Atlantic  City  FIAO  crashed  near  Oak  Grove,  Pennsyl¬ 
vania,  killing  all  three  crew  members.  Based  on  their 
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investigation,  the  NTSB  determined  that  the  aircraft 
had  entered  an  area  of  moderate  icing  conditions  and 
was  in  a  holding  pattern  while  the  crew  checked  on 
flight  inspection  equipment  malfunctions.  The  crew 
apparently  noticed  ice  accretion  and  activated  the 
surface  de-icing  system.  Ice  broke  lose  and  entered 
engine  intakes,  causing  both  engines  to  flame  out. 
During  emergency  descent,  the  crew  attempted  to 
restart  engines  with  no  success.  In  addition,  both 
pilots  were  reported  to  have  experienced  stress  in  their 
private  lives  that  could  also  have  affected  their  perfor¬ 
mance  (NTSB,  1994). 

An  earlier,  non  fatal  accident  occurred  on  Septem¬ 
ber  29,  1986,  when  a  Rockwell  Sab  reliner  (NA-265) 
operated  by  the  Oklahoma  City  FIAO  was  destroyed 
while  landing  near  Liberal,  Kansas.  The  NTSB  deter¬ 
mined  that  the  flying  pilot  (in  this  case  the  SIC) 
touched  down  21  feet  short  of  the  runway.  Due  to  the 
impact  the  landing  gear  collapsed  and  the  aircraft 
traveled  the  full  length  of  the  runway  before  stopping 
on  a  golf  course  a  quarter  of  a  mile  away.  Also  cited  as 
contributing  to  the  accident  was  the  PIC’s  failure  to 
follow  proper  flight  procedures.  (NTSB,  1994). 

All  three  of  the  above  accidents  contained  elements 
of  ineffective  crew  resource  management.  Thus,  the 
accident  reports  themselves,  served  as  one  means  of 
identifying  CRM  training  needs.  In  addition  to  acci¬ 
dent  reports,  CRM  training  needs  were  also  identified 
by  analyzing  the  results  of  safety  meeting  discussions 
following  the  N82  accident,  and  by  asking  subject 
matter  experts  to  provide  narratives  of  problematic 
situations  that  they  had  encountered  while  perform¬ 
ing  a  flight  inspection  mission.  This  report  summarizes 
the  training  needs  derived  from  the  last  two  sources. 

METHOD 

Participants 

Subjects  matter  experts  (SMEs)  were  recruited  dur¬ 
ing  a  one-day  post  accident  CRM  awareness  training 
course  conducted  at  each  of  the  FIAO.  Training 
participants  were  advised  that  they  could  participate 
individually  in  the  data  collection  for  the  training 
needs  analysis,  or  they  could  provide  their  input  to 
their  FIAO’s  CRM  representative.  Fifty-eight  flight 
inspection  SMEs  chose  to  provide  their  individual 


input.  This  represented  30%  of  the  flight  inspection 
workforce.  Subjects  consisted  of  PICs,  SICs,  and 
electronics  technicians.  To  protect  the  anonymity  of 
the  individuals  and  their  respective  FIAOs,  no  demo¬ 
graphic  data  were  collected.  In  addition,  all  surveys 
were  destroyed  following  data  entry  and  analysis. 
These  measures  were  taken  to  assure  the  participants 
that  they  could  be  candid  with  their  responses,  and 
that  no  punitive  action  could  result  from  their  partici¬ 
pation  in  the  survey. 

Instruments 

Two  written  questionnaires  were  used  to  collect 
data  for  the  training  needs  analysis.  The  purpose  of 
the  first  questionnaire  (Safety  Survey)  was  to  identify: 
(1)  CRM  dimensions  that  needed  to  be  addressed  in 
awareness  training;  and  (2)  the  organizational  factors 
that  needed  to  be  addressed  to  support  a  long  term 
CRM  effort.  The  Safety  Survey  presented  SMEs  with 
109  issues,  extracted  verbatim  from  written  summa¬ 
ries  of  the  November  1 993  safety  meeting  discussions. 
For  each  safety  issue,  SMEs  indicated  which  of  13 
performance  categories  most  applied  (see  Table  1). 
Multiple  categories  could  be  assigned  to  a  given  safety 
issue.  Definitions  for  the  first  10  categories  were 
derived  from  commonly  accepted  CRM  dimensions 
(Bowers,  Morgan,  Salas,  &c  Converse,  1993;  FAA, 
1993).  These  categories  represented  potential  CRM 
awareness  training  modules.  Categories  11  and  12 
(skill  proficiency  and  organizational  factors  )  were 
included,  based  on  the  literature  previously  reviewed. 
Category  13  (CRM  Dimension  Not  Specified)  was 
included  for  completeness. 

Two  kinds  of  analysis  were  planned  for  the  Safety 
Survey.  First,  safety  issues  would  be  sorted  by  the 
performance  categories  to  which  they  most  applied. 
Second,  cluster  analytic  techniques  would  be  used  to 
determine  the  hierarchical  structure  of  the  safety  is¬ 
sues  and  how  they  related  to  overall  mission  success. 
Whereas  the  first  analysis  would  identify  specific 
training  needs,  the  second  analysis  would  provide  a 
broader  structural  context  for  interpreting  those  needs. 

The  purpose  of  the  second  questionnaire  (Incident 
Survey)  was  to  identify  the  phases  of  flight  that  were 
most  problematic  from  a  CRM  perspective,  as  well  as 
to  provide  insight  concerning  the  cause  of  those 
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TABLE  1 

Performance  Categories 


1 .  Mission  Anaiysis*  -  includes  monitoring,  allocating,  and  coordinating  the  resources  of  the  crew  and  the 
aircraft:  prioritizing  tasks;  setting  goals  and  developing  plans  to  accomplish  the  goals;  and  creating 
contingency  plans. 

2.  Situational  Awareness*  -refers  to  identifying  the  source  and  nature  of  problems,  maintaining  an  accurate 
perception  of  the  aircraft’s  location  relative  to  the  external  environment,  and  detecting  situations  that 
require  action. 

3.  Decision  Making*  -  includes  identifying  possible  solutions  to  problems,  evaluating  the  consequences  of 
each  alternative,  selecting  the  best  alternative,  and  gathering  information  needed  prior  to  arriving  at  a 
decision. 

4.  Communication*  -  includes  sending,  receiving,  and  acknowledging  information  among  crew  members  in 
a  way  that  facilitates  the  accurate  transfer  of  information 

5.  Crew  interpersonal  Climate  (CC)**  -  Refers  to  the  overall  interpersonal  atmosphere  of  the  crew.  It 
includes  the  way  interpersonal  conflicts  are  resolved,  the  degree  to  which  members  enjoy  working 
together,  the  shared  values  they  have  about  their  profession,  and  the  degree  of  comfort  the  crew  has  with 
the  way  crew  activities  are  coordinated. 

6.  Leadership*  refers  to  directing  the  activities  of  others,  monitoring  and  assessing  the  performance  of  crew 
members,  motivating  members,  and  communicating  mission  information. 

7.  Adaptability*  -  refers  to  the  ability  to  alter  one’s  course  of  action  as  necessary,  maintain  constructive 
behavior  under  pressure,  and  adapt  to  internal  or  external  changes. 

8.  Assertiveness*  -  refers  to  the  willingness  to  make  decisions,  demonstrating  initiative,  and  maintaining 
one’s  position  until  convinced  otherwise  by  facts. 

9.  Workload  Management***  -  refers  to  the  ability  to  schedule,  structure,  and  coordinate  mission  activities 
so  as  not  to  jeopardize  situational  awareness  during  any  phase  of  the  mission. 

1 0.  Life  Stress***  -  refers  to  stressors  outside  of  the  context  of  work  which  interfere  with  a  person’s  ability  to 
perform  as  expected  by  self  and  crew  members. 

11.  Skill  Proficiency-  refers  to  technical  skills,  the  absence  of  which  adversely  impacts  the  crew’s  confidence 
in  their  ability  to  carry  out  the  mission. 

12.  Organizational  Factors  -  refers  to  formal  (regulations)  and  informal  (imposed  by  a  given  manager) 
policies  and  procedures  which  constrain  the  way  a  crew  should  ideally  function. 

13.  CRM  Not  Specified  -  refers  to  any  CRM  principle  not  specified  in  the  above  list. 


*  After  Bowers,  et  al.  (1993)  definitions  of  crew  coordination  demands. 

**  After  Feldman  (1968),  3-factor  model  of  cohesion. 

***  After  FAA  Crew  Resource  Management  Training  Advisory  Circular,  120-51 A  (1993). 
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problems.  SMEs  were  given  written  instructions  ask¬ 
ing  them  to  describe  problematic  situations  that  they 
had  experienced  while  performing  a  flight  inspection. 
The  instructions  were  modeled  after  Flanagan’s  ( 1 954) 
critical  incidence  methodology,  commonly  used  to 
identify  behaviors  that  differentiate  between  varying 
levels  of  performance.  In  describing  their  situations, 
SMEs  were  instructed  to  include:  (1)  the  specific  prob¬ 
lem  that  was  encountered,  (2)  the  phase  of  the  flight  in 
which  the  problem  occurred,  (3)  what  led  up  to  the 
incident,  (4)  what  crew  members  did  that  was  effective  or 
ineffective,  and  (5)  the  impact  that  the  problem  had  on 
the  success  of  the  overall  mission.  Data  from  this  survey 
were  subject  to  content  analyses  by  the  authors. 

Procedures 

Both  the  Safety  and  Incident  surveys  were  admin¬ 
istered  during  the  one  day  post  accident  CRM  aware¬ 
ness  training  conducted  at  each  of  the  FIAOs.  The 
training  seminar  had  a  five  fold  purpose:  ( 1 )  to  present 
an  overview  of  what  was  meant  by  CRM,  (2)  to 
outline  the  steps  that  would  be  used  to  develop  a  three 
day  flight  inspection  CRM  awareness  course,  (3)  to 
establish  trust  and  rapport  with  the  flight  inspection 
crews,  (4)  to  deliver  a  training  module  on  aeronautical 
decision  making,  (5)  to  conduct  small  group  discus¬ 
sions  after  viewing  an  aircraft  accident  scenario  simi¬ 
lar  to  the  N82  accident.  After  outlining  the  CRM 
course  development  process,  time  was  set  aside  for 
volunteers  to  participate  in  the  identification  of  CRM 
training  needs..  Volunteers  were  briefed  on  the  pur¬ 
pose  of  the  training  needs  analysis  and  the  part  it 
would  play  in  the  course  development  process.  After¬ 
wards,  a  question  and  answer  period  was  conducted  to 
alleviate  any  doubts  or  concerns  participants  had 
about  the  two  surveys.  Verbal  and  written  instruc¬ 
tions  were  then  provided  before  administering  each 
survey.  A  survey  administrator  was  present  to  answer 
any  additional  questions.  No  time  limit  was  imposed. 

RESULTS 

Safety  Survey 

The  first  step  in  analyzing  the  Safety  Survey  was  to 
construct  a  109  by  13  frequency  matrix  in  which  the 
rows  contained  the  109  issues  extracted  from  the 


safety  meetings,  and  the  columns  contained  the  13 
performance  categories  identified  in  Table  1.  Cell 
values  represented  the  frequency  with  which  a  given 
safety  issue  was  matched  to  a  particular  category.  The 
maximum  value  for  any  cell  was  58,  corresponding  to 
the  number  of  people  who  participated  in  the  survey. 
From  this  frequency  matrix,  each  column  was  sorted 
based  on  cell  values.  This  produced  a  priority  listing 
of  safety  issues  associated  with  each  training  category. 
The  results  of  this  analysis  appear  in  Table  2. 

In  the  first  column  of  Table  2,  training  categories 
are  presented  in  descending  order  based  on  the  fre¬ 
quency  with  which  they  were  endorsed.  The  percent¬ 
ages  of  endorsement  are  indicated  in  parenthesis. 
Over  half  of  the  problems  discussed  during  the  safety 
meetings  concerned  problems  associated  with  man¬ 
agement  (20%),  crew  interpersonal  climate  (11%), 
situational  awareness  (10%),  and  leadership  (10%). 
Decision  making,  life  stress,  adaptability,  and 
assertiveness  together  accounted  for  the  bottom  11% 
of  the  endorsements.  Presented  in  the  second  column 
of  Table  2  are  the  training  needs  that  were  identified 
for  each  training  category.  These  include  both  train¬ 
ing  for  management  to  support  CRM  (indicated  un¬ 
der  Organizational  Factors)  as  well  as  training  for 
crew  members  themselves.  Notice  that  no  training 
needs  were  identified  under  the  “assertiveness”  cat¬ 
egory.  This  does  not  suggest  that  there  is  no  need  for 
assertiveness  training.  It  simply  means  that  survey 
respondents  did  not  indicate  that  assertiveness  was 
related  to  the  issues  raised  during  the  safety  meetings. 

T  o  determine  the  hierarchical  structure  of  the  Safety 
Survey,  the  frequency  matrix  was  converted  into  a 
proximity  matrix,  using  squared  Euclidean  distances 
as  a  measure  of  similarity.  Clusters  were  then  formed 
using  Ward’s  method  in  SPSS  for  Windows  version 
6.0.  Ward’s  method  forms  subgroupings  (clusters) 
based  on  maximizing  the  ratio  of  between-cluster 
variance  over  within-cluster  variance.  The  resulting 
dendogram  depicting  the  hierarchical  relationship  of 
five  interpretable  clusters  is  shown  in  Figure  1 .  These 
clusters  include:  (1)  technical  skills,  (2)  organiza¬ 
tional  stressors,  (3)  crew  stressors,  (4)  situational 
awareness,  and  (5)  planning  and  decision  making. 
Training  themes  associated  with  each  of  five  clusters 
are  shown  in  Table  3. 
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TABLE  2 

Crew  Resource  Management  Training  Needs 

Performance  Categories  Ranked  by  Training  Needs 

Percentage  of  Endorsement  (%) _ 

Organizational  Factors  (20%)  Need  for  management  to:  "walk  the  talk;"  develop  and  enforce  well 

defined  poiicies  and  procedures;  understand  the  nature  of  the  flight 
inspection  mission  and  how  managerial  performance  affects  crew 
performance;  establish  standardization;  and  create  an  atmosphere  in 
which  candid  exchanges  about  safety  can  occur. 

Crew  Interpersonal  Climate  (1 1  %)  Need  for  the  entire  crew  to  participate  in  creating  a  safe  flying 

environment;  to  be  clear  on  their  respective  team  roles;  and  to  resolve 
interpersonal  conflicts  before  flying. 

Situational  Awareness  (10%)  Need  for  the  entire  crew  to  participate  in  altitude  and  aircraft  position 

awareness. 

Leadership  (1 0%)  Need  for  the  Pilot  In  Command  to  clarify  roles  each  crew  member  will 

play  prior  to  flying  the  mission;  and  to  distribute  the  workload  so  that 
the  PIC  is  not  over  tasked. 

Communications  (9%)  Need  for  a  pre-flight  briefing  highlighting  the  major  phases  of  the 

mission  so  that  the  entire  crew  is  prepared,  and  the  need  to  create  a 
professional  climate  for  open  communication  exchanges  among  crew 
members. 

Mission  Analysis  (8%)  Need  to  include  organizational  factors  affecting  the  mission,  in 

particular,  the  scheduling  of  flight  inspections,  the  availability  of  aircraf 
and  the  conduct  of  pre-flight  briefings 

Skill  Proficiencies  (8%)  Need  to  maintain  currency  and  the  need  to  train  to  standardization. 

Workload  Management  (7%)  Need  to  coordinate  with  air  traffic  control,  and  to  reduce  scheduling 

pressures. 

CRM  Dimension  not  specified  (6%)  No  training  needs  identified. 

Decision  Making  (4%)  Need  for  the  crew  to  take  charge  of  decisions  affecting  the  success  of 

the  mission,  in  particular  deciding  on  whether  or  not  it  is  safe  to  fly. 

Life  Stress  (3%)  Need  to  quickly  resolve  organizational  uncertainty  (office  closings  and 

relocations). 

Adaptability  (2%)  Need  to  adjust  to  flight  check  restrictions  imposed  by  air  traffic  control 

without  creating  an  adversarial  situation,  and  to  adapt  to  itinerary 
changes  rather  than  being  distracted  by  them. 

Assertiveness  (2%)  No  training  needs  identified. 
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Figure  1  shows  that  mission  success  consists  of  two 
clusters,  one  that  deals  with  Technical  Performance, 
and  one  that  deals  with  Crew  Participation.  Technical 
performance  is  further  divided  into  issues  related  to 
technical  flying  skills,  as  well  as  stressors  that  act  to 
interfere  with  the  performance  of  those  skills.  This 
interference  consists  of  factors  residing  within  the 
organization  and  factors  that  reside  within  the  crews. 
Crew  participation  is  comprised  of  issues  related  to 
maintaining  situational  awareness,  and  planning  and 
decision  making. 

Table  3  shows  that  the  training  needs  that  emerge 
from  the  cluster  analysis  may  be  further  divided  into 
two  categories:  (1)  areas  of  personal  concern,  and  (2) 
areas  of  personal  control.  This  division  is  also  shown 
in  Figure  1  by  the  dashed  line.  Contained  in  areas  of 
personal  concern  are  those  issues  that  concern  flight 
inspection  crews,  such  as  technical  skills  and  organi¬ 
zational  stressors,  but  are  outside  their  personal  con¬ 
trol,  For  example,  flight  crews  can  request  changes  in 
the  kind  of  training  that  they  receive,  however,  they 
do  not  have  the  power  to  make  those  changes.  In 
contrast,  areas  of  personal  control,  such  as  crew  stressors. 


situational  awareness,  planning,  and  decision  mak¬ 
ing,  are  more  strongly  associated  with  factors  that 
crew  members  themselves  have  the  power  to  change. 

Incident  Survey 

In  analyzing  the  results  of  the  Incident  Survey,  a 
problem  arose  due  to  incomplete  data.  Of  the  five 
factors  that  SMEs  were  asked  to  include  in  their 
narratives,  only  the  first  two  were  reported  with  any 
degree  of  clarity.  Subsequent  discussions  with  the 
SMEs  revealed  they  were  more  comfortable  talking 
about  problems  than  writing  about  them.  Because  of 
problems  with  incomplete  data,  the  results  of  the 
content  analyses  are  reported  for  heuristic  purposes 
only,  and  should  not  be  construed  as  representing 
scientific  rigor. 

Data  from  the  CRM  Incident  Survey  were  coded  by 
the  authors  according  to  the  phase  of  the  mission  in 
which  problems  occurred,  as  well  as  the  most  probable 
cause  of  the  problem  (i.e.,  weather,  person,  or  equip¬ 
ment  failure).  This  information  is  provided  in  the  first 
four  columns  of  Table  4.  The  numbers  recorded  in 
each  cell  refer  to  the  frequency  with  which  a  probable 


TABLE  3 

Cluster  Themes 


Clusters 


Dominant  Themes 


Areas  of  Concern 

Technical  Skills  T rain  to  a  standard,  and  maintain  currency  with  equipment 

Organizational  Stressors  Mission-safety  conflict,  aircraft  maintenance  problems,  and 

lack  of  organizational  support 


Areas  of  Control 

Crew  Stressors  Role  conflict,  role  ambiguity,  low  morale 

Situational  Awareness  Altitude  and  position  awareness 


Planning  and  Decision  Preflight  briefings,  aircraft  safety 

Making 


TABLE  4 

Probable  Cause  of  Problems  Encountered  at  Each  Phase  of  the  Flight  Inspection  Mission 


Mission  Phase 

Number  of 
People 
Problems 

Number  of 
Weather 
Problems 

Number  of 
Equipment 
Problems 

Row  Frequencies 

Row% 

Pre-Departure 

4 

0 

0 

4 

6.9% 

Take-off 

8 

0 

2 

10 

Enroute 

5 

5 

3 

13 

Inspection 

20 

3 

2 

25 

Landing 

2 

2 

2 

6 

10.3% 

39 

10 

9 

58 

Column  % 

67.2% 

17.3% 

15.5% 

100% 
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cause  occurred  within  a  given  phase  of  the  mission.  In 
addition.  Table  4  also  shows  column  and  row  fre¬ 
quencies  and  percentages. 

The  most  probable  cause  of  the  problems  reported 
in  Table  4  was  associated  with  people-related  issues 
(67.2%)  involving  interactions  among  crew  members 
and  with  air  traffic  control  and  airway  facilities  ground 
maintenance.  This  was  followed  by  weather-related 
problems  (17.3%),  and  equipment  malfunctions 
(15.5%).  Most  of  the  incidents  occurred  while  per¬ 
forming  the  flight  inspection  (43.2%),  followed  by 
enroute  (22.4%),  take-off  (17.2%),  landing  (10.3%), 
and  pre-departure  (6.9%).  A  content  analysis  of  the 
people-related  problems  suggested  three  problematic 
areas  involving:  (1)  errors  in  information  processing 
(e.g.,  situational  awareness,  and  decision  making);  (2) 
errors  in  information  transfer  (i.e.,  communications); 
and  (3)  factors  associated  with  the  way  crew  members 
related  to  each  other  (e.g.,  interpersonal  climate). 

A  second  content  analysis  of  the  CRM  incidents 
was  conducted  to  identify  the  CRM  performance 
dimensions  associated  with  a  given  incident.  Follow¬ 


ing  the  same  procedures  used  by  SMEs  in  the  Safety 
Survey,  the  authors  identified  the  performance  cat¬ 
egories  (see  Table  1)  that  most  applied.  Percentages  of 
incidents  associated  with  a  given  performance  cat¬ 
egory  were  then  derived  as  reported  for  the  safety 
survey.  This  information  is  presented  in  column  two 
of  Table  5.  For  comparison  purposes,  the  results  of  the 
Safety  Survey  are  also  shown  in  column  3.  For  the 
most  part,  the  rank  order  (based  on  percentage  values) 
of  the  CRM  categories  are  the  same  for  both  surveys. 
Notice,  however,  that  there  is  an  extreme  difference 
associated  with  two  CRM  categories,  decision  making 
and  organizational  factors.  Whereas  decision  making 
was  a  significant  factor  (22%)  associated  with  the 
CRM  incidents,  it  played  a  relatively  minor  role  in  the 
CRM  safety  issues  (4%).  Similarly,  organizational 
factors  were  the  dominant  issues  discussed  in  the 
CRM  safety  survey  (20%),  but  they  contributed 
little  (1%)  to  the  factors  associated  with  the  CRM 
incidents.  These  differences  have  important  impli¬ 
cations  for  CRM  training  and  will  be  discussed 
later  in  this  report. 


TABLE  5 

Performance  Category  Comparisons  between  Incident  Survey  and  Safety  Survey 


Performance  Categories 

Incident  Survey 
Percentage  of 
Occurrences 

Safety  Survey 
Percentage  of 
Endorsements 

Decision  Making 

22% 

4% 

Crew  Interpersonal  Climate 

22% 

11% 

Situational  Awareness 

20% 

10% 

Communications 

15% 

9% 

Mission  Anaiysis 

8% 

8% 

Assertiveness 

7% 

2% 

Leadership 

3% 

10% 

Workioad  Management 

2% 

7% 

Organizationai  Factors 

1% 

20% 

Skili  Proficiencies 

0% 

8% 

Life  Stress 

0% 

3% 

Adaptability 

0% 

2% 

CRM  Dimension  not  specified 

0% 

6% 
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DISCUSSION 

The  cluster  analytic  results  of  the  training  needs 
analysis  support  earlier  findings  that  flight  crew  per¬ 
formance  is  dependent  on  three  factors:  (1)  technical 
skills,  (2)  resource  management  skills,  and  (3)  the 
organizational  context  in  which  flight  crews  operate. 
As  shown  in  Figure  1 ,  technical  performance  and  crew 
participation  form  two  distinct  classifications  of  train¬ 
ing  needs,  with  organizational  and  crew  contextual 
factors  acting  as  stressors  that  interfere  with  the  tech¬ 
nical  performance  of  flight  inspection  crews.  Further¬ 
more,  the  training  needs  that  emerge  from  this 
classification  may  be  further  divided  into  two  catego¬ 
ries:  (1)  those  factors  over  which  flight  crews  have 
control,  and  (2)  factors  that  concern  flight  crews,  but 
control  of  which  resides  within  the  organization. 
Using  the  structural  framework  of  Figure  1,  several 
training  implications  are  especially  worth  noting. 

First,  crew  members  reported  problems  with  the 
technical  training  they  received.  In  particular,  pilots 
complained  that  some  of  them  were  not  getting  enough 
flying  time  which  made  them  feel  not  as  technically 
proficient  as  they  would  have  liked.  In  addition,  pilots 
were  not  always  checked  out  on  equipment  modifica¬ 
tions  prior  to  conducting  a  flight  inspection  mission. 
Since  the  single  most  important  resource  that  crew 
members  possess  is  technical  skill,  technical  training 
deficiencies,  such  as  these,  must  first  be  addressed  for 
crew  resource  management  training  to  have  a  positive 
effect  on  crew  performance. 

Second,  the  results  of  the  needs  analysis  suggested 
that  crews  would  benefit  by  more  active  crew  partici¬ 
pation,  particularly  with  regard  to  three  areas:  (1)  pre¬ 
mission  briefings,  (2)  decisions  about  safety,  and  (3) 
maintaining  aircraft  situational  awareness.  The  im¬ 
portance  of  a  pre-mission  briefing  cannot  be  over 
emphasized.  It  is  during  this  briefing  that  crews  de¬ 
velop  what  Cannon-Bowers,  et  al.  (1993)  and  others 
have  called  a  shared  mental  model  of  the  mission  (for 
a  review  of  the  literature  see  Klimoski  &  Mohammed, 
1994).  A  shared  mental  model  may  be  thought  of  as  a 
common  set  of  expectations  about  what  will  occur 
during  a  mission.  Included  in  this  mental  model  are 
expectations  concerning  the  time-sequencing  of  mis¬ 
sion  events,  the  tasks  to  be  performed,  and  how 


individual  efforts  will  be  coordinated  (Cannon-Bow¬ 
ers,  et  al.,  1993).  When  a  pre-mission  briefing  is 
lacking,  crew  members  must  rely  on  past  experiences 
to  guide  their  performance.  The  implicit  assumption 
is  that  everyone  is  operating  with  the  same  set  of 
expectations.  Unfortunately,  it  is  usually  under  non¬ 
routine  conditions  that  the  fallacy  of  this  assumption 
surfaces  (Orasanu,  1994). 

In  addition  to  establishing  a  shared  mental  model 
of  the  mission,  the  pre-mission  briefing  is  an  excellent 
time  to  address  the  leadership,  communications,  and 
crew  climate  concerns,  as  indicated  in  Table  2.  How 
a  PIC  conducts  a  pre-mission  briefing  sets  the  stage 
for  the  communication  patterns  that  will  emerge 
among  crew  members  (Hackman,  1993).  If  the  PIC 
provides  a  well  organized  briefing  and  solicits  input, 
then  he  or  she  establishes  an  atmosphere  of  profes¬ 
sional  competency  in  which  crew  members  feel  free  to 
voice  concerns.  Furthermore,  to  the  extent  that  crews 
can  resolve  differences  of  opinions  prior  to  flight,  they 
are  less  likely  to  be  distracted  by  those  differences 
during  the  course  of  the  mission.  By  involving  all 
three  crew  members  (PIC,  SIC,  and  ET)  in  decisions 
regarding  flight  safety,  crews  create  a  climate  in  which 
flight  safety  is  a  shared  responsibility.  The  need  for 
such  involvement  is  reflected  in  the  decision  making 
and  situational  awareness  CRM  training  needs  pre¬ 
sented  in  Table  2. 

A  third  implication  derived  from  the  needs  analysis 
concerns  the  effects  that  organizational  and  crew 
stressors  have  on  the  technical  performance  of  a  flight 
inspection  mission.  Crew  stressors,  such  as  role  ambi¬ 
guity  and  role  conflict,  have  already  been  addressed  as 
issues  that  should  be  clarified  during  a  pre-mission 
briefing.  Organizational  stressors,  however,  require 
further  elaboration. 

Flight  inspection  crews  are  mission  oriented.  Their 
job  is  to  fly  the  aircraft  over  a  prescribed  course  under 
the  guidance  of  ground-to-air  flight  navigation  equip¬ 
ment,  while  onboard  electronic  equipment  records 
the  aircraft’s  relative  position  in  space.  The  ground 
facility  is  certified  when  the  recorded  data  indicate  the 
aircraft  flew  the  desired  course  within  specific  toler¬ 
ances.  Facilities  must  be  flight  checked  and  certified 
prior  to  use  when  ground  navigation  aids  or  equip¬ 
ment  are  installed,  repaired,  removed,  or  replaced  and 
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when  required  periodic  inspections  are  due.  Naviga¬ 
tional  aids  may  also  be  flight  checked  following  air¬ 
craft  accidents. 

Due  to  a  variety  of  reasons  (such  as  a  facility  outage 
at  O’Hare  International,  a  high  density  traffic  air¬ 
port),  there  can  be  increasing  pressure  on  flight  crews 
to  perform  flight  checks  during  marginal  weather  or 
during  off  peak  traffic  hours  late  at  night  when  crews 
tend  to  be  fatigued.  Crews  consider  these  conditions 
to  be  unsafe,  and  problems  can  arise  when  flight  crews 
perceive  (correctly  or  incorrectly)  that  their  manage¬ 
ment  is  more  concerned  about  getting  the  job  done 
than  they  are  about  their  safety. 

In  a  study  conducted  by  Witt,  Hilton,  &  Helman 
(1994),  workplace  safety  perceptions  were  found  to  be 
directly  related  to  perceived  managerial  support  for 
safety.  Moreover,  management  was  perceived  to  be 
most  supportive  of  safety  when  employees  experi¬ 
enced  a  work  climate  that  was  “relatively  free  from 
politics  and  unfairness,  and  in  which  everyone  shared 
a  common  view  regarding  the  importance  of  safety.” 
The  authors  concluded  that  how  managers  deal  with 
everyday  problems  sets  the  tone  for  how  they  will  deal 
with  safety  related  issues.  This  conclusion  is  further 
supported  by  the  organizational  factor  concerns  re¬ 
ported  in  Table  2.  In  particular,  flight  crews  expressed 
the  need  for  management  to  understand  how  manage¬ 
rial  performance  affects  crew  performance,  as  well  as 
the  need  for  management  to  create  an  atmosphere  in 
which  candid  exchanges  about  safety  can  occur. 

Concerns  about  organizational  stressors  are  valid 
and  need  to  be  addressed  by  the  organization;  how¬ 
ever,  caution  is  advised  when  addressing  these  issues 
during  CRM  awareness  training.  The  inclusion  of 
organizational  factors  is  likely  to  shift  the  focus  of 
CRM  training  from  what  Covey  (1989)  calls  “areas  of 
control”  to  “areas  of  concern.”  Areas  of  concern  are 
those  issues  in  which  crews  have  a  vested  interested, 
but  little  personal  control  over  the  outcome.  In  con¬ 
trast,  areas  of  control  represent  issues  in  which  the 
crews,  themselves,  have  the  power  to  effect  change. 
Covey  further  notes  that  there  is  a  tendency  for  people 
to  spend  a  considerable  amount  of  time  attempting  to 
address  areas  of  concern  while  neglecting  areas  of 
control.  This  division  is  shown  in  Figure  1  and  is 
reflected  in  the  frequency  that  organizational  factors 


were  discussed  during  the  safety  meetings.  Of  all 
CRM  categories,  organizational  factors  was  the  most 
frequently  discussed,  but  it  is  the  area  over  which 
crews  have  the  least  personal  control. 

One  way  of  keeping  CRM  awareness  training  fo¬ 
cused  on  areas  of  control  is  to  develop  training  sce¬ 
narios  around  situations  that  crews  have  actually 
experienced  while  in  flight.  This  can  be  seen  by  the 
Table  5  comparison  between  the  rank  ordering  of 
CRM  categories  derived  from  the  Safety  Survey,  and 
those  derived  from  the  CRM  Incidents.  Whereas 
organizational  factors  represented  20%  of  the  issues 
discussed  in  the  safety  meetings,  they  represented  only 
1%  of  the  CRM  incidents  reported  by  flight  inspec¬ 
tion  crews. 

The  above  results  further  suggest  that  by  simply 
changing  the  field  of  reference  from  problems  in 
general  (as  discussed  in  the  November  1993  safety 
meetings)  to  problems  in  the  air  (as  reported  in  the 
Incident  Survey),  crew  members  are  more  apt  to  take 
personal  responsibility  for  their  actions.  This  pro¬ 
jected  outcome  forms  the  basis  of  the  practice  and 
feedback  phase  of  CRM  training  (FAA,  1993).  In  this 
phase,  instead  of  evaluating  the  performance  of  others 
(as  in  watching  accident  recreations),  crews  are  video¬ 
taped  while  performing  under  mission  simulated  con¬ 
ditions.  Afterwards,  crews  view  the  video  playback  of 
their  performance  and  critique  themselves  on  their 
resource  management  skills.  Factors  affecting  crew 
performance  are  often  self-evident,  however,  a  trained 
facilitator  may  be  necessary  to  ensure  that  crews  per¬ 
form  a  self  critique  on  more  than  just  their  technical 
performance  (Butler,  1993). 

Finally,  in  support  of  developing  CRM  awareness 
training,  data  from  the  CRM  incident  survey  (see 
Table  4)  suggest  that  most  flight  inspection  CRM- 
related  problems  (43.2%)  occurred  while  performing 
aircraft  maneuvers  during  the  airborne  testing  phase 
of  flight  (i.e.,  the  actual  flight  inspection).  These 
problems  involve  communication  errors,  errors  in 
information  processing,  and  factors  associated  with 
the  crew’s  interpersonal  climate.  Although  generic 
CRM  training  materials  in  those  areas  exist,  the  FAA 
CRM  Advisory  Circular  (120-51  A)  cautions  against 
using  such  material  without  first  adapting  it  to  reflect 
the  nature  and  needs  of  the  organization. 
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The  most  successful  CRM  training  programs  are 
those  which  are  internally  developed  (Captain  J.  E. 
Carroll,  retired  United  Airlines  Vice  President  of 
Safety  and  Training,  personal  communications.  May 
17, 1994).  This  is  because  crew  resource  management 
is  more  than  just  a  course,  it  is  a  philosophy  or  way  of 
knowing  and  thinking  about  flying  a  mission.  This 
philosophy  is  manifested  in  an  organization’s  culture. 
By  developing  an  internal  course,  organizations  must 
address  such  questions  as:  (1)  what  is  the  purpose  of 
the  flying  mission  and  what  are  its  priorities,  (2)  how 
do  organizational  policies  and  practices  affect  the 
flying  mission,  (3)  what  really  takes  place  when  a 
flight  crew  flies  a  mission,  (4)  how  prepared  are  flight 
crews  for  accomplishing  their  mission,  and  (5)  what 
habits  have  flight  crews  developed  that  enhance  or 
impede  the  success  of  a  mission?  When  a  generic 
course  is  used  to  deliver  CRM  training,  there  is  a 
tendency  for  organizations  to  side-step  many  of  these 
harder  questions,  and  instead  just  implement  a  course 
as  a  “quick  fix.”  As  a  consequence,  such  CRM  pro¬ 
grams  tend  to  fade  over  time. 

There  is  a  down  side  to  developing  a  CRM  pro¬ 
gram,  however,  when  internal  resources  are  used:  it 
takes  time,  and  mistakes  will  be  made.  Thus,  unless  an 
organization  is  willing  to  commit  the  necessary  re¬ 
sources  to  improve  upon  initial  efforts,  its  CRM 
program  is  likely  to  fall  below  expectations.  In  addi¬ 
tion,  flight  crews  should  understand  that  their  CRM 
program  will  evolve  over  time,  with  its  success  depen¬ 
dent  upon  the  degree  to  which  flight  crews  are  actively 
involved  in  course  development,  program  implemen¬ 
tation,  and  program  evaluation.  For  these  reasons,  the 
following  recommendations  are  made:  (1)  develop  a 
CRM  awareness  course  from  within  the  FAA,  rather 
than  trying  to  fit  AVN  to  a  generic  CRM  course;  (2) 
create  a  CRM  program  that  can  be  sustained  over  the 
long  term,  rather  than  attempting  to  implement  a 
“quick  fix;”  (3)  involve  the  flight  crews  in  the  course 
development  process;  (4)  develop  a  common  language 
between  the  flight  crews  and  AVN  management  for 
discussing  CRM  issues;  (5)  develop  a  mission-based 
flight  inspection  simulation  training  environment  for 
the  practice  and  feedback  phase  of  CRM  training;  and 
(6)  adopt  organizational  reinforcers  to  support  the 
principles  of  CRM. 
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